1. Field of the Invention
The present invention relates to a stream data control server, a stream data control method, and a stream data controlling program, which cause a predetermined processing unit such as a storage unit and a filter unit to process data which is continuously generated.
2. Description of the Related Art
As technologies for storing data in storage units, for instance, JP 2003-006005 A has disclosed a method in which a data storage unit equipped with a plurality of storage media stores data in a storage medium having higher security according to a data priority of the data. FIG. 15 is a block diagram showing the data storage unit described in JP 2003-006005 A as a structural example. The data storage unit described in JP 2003-006005 A has been equipped with rule storage unit 91, data-priority calculating unit 92, storing destination selecting unit 95, security storage unit 96, and storing/migrating process unit 97.
In the above-mentioned configuration, the rule storage unit 91 stores therein a rule which gives orders to data based upon data priorities of the data, and the data-priority calculating unit 92 gives orders with respect to data stored in the storage unit based upon the rule stored in the rule storage unit 91. The security storage unit 96 stores therein a rule which gives orders to the plurality of storage media based upon security thereof, and also, the storing destination selecting unit 95 gives orders to the plurality of storage media employed in the storage unit based upon the rule stored in the security storage unit 96. The storing/migrating process unit 97 selects such a storage medium to which data should be stored from the plurality of storage media employed in the storage unit based upon the orders given to the data and the orders given to the storage media, and then, stores the data in the selected storage medium.
Also, in “An optimization method for multiple persistency requirements on stream management system” by Yamada et al. 2007, Japanese Electronic Information Communication Institute Data Engineering Workshop (DEWS), a method for realizing a plurality of storage processings with respect to data such as logs, which is generated in a continuous manner has been disclosed. FIG. 16 is a block diagram showing a stream management system as a structural example in “An optimization method for multiple persistency requirements on stream management system” by Yamada et al. 2007, Japanese Electronic Information Communication Institute Data Engineering Workshop (DEWS). The stream management system described in “An optimization method for multiple persistency requirements on stream management system” by Yamada et al. 2007, Japanese Electronic Information Communication Institute Data Engineering Workshop (DEWS) is equipped with a requirement description analyzing unit 81, an optimizing unit 82, a stream processor 83, a stream wrapper 84, a stream archiver 85, and also, a DBMS connector 86.
In the above-mentioned configuration, the requirement description analyzing unit 81 receives a data process request issued from a user, and then, transfers the received data process request to the optimizing unit 82. The optimizing unit 82 arranges the respective processings to the stream processor 83 and the stream archiver 85. The stream processor 83 accesses data which has been generated from a stream type information source stored via the stream wrapper 84, and also accesses data which has been stored via the stream archiver 85 so as to execute the processings arranged in the optimum manner. Then, the stream processor 83 returns the processed result to the user, or transfers the processed result to the stream archiver 85 in such a manner that the processed data is stored. The stream archiver 85 inputs and outputs stored data (archived data) via the DBMS connector 86 with respect to one or more pieces of DBMSs in accordance with the processings arranged in the optimum manner.
The above-mentioned conventional technologies have the following problem that, as a first problem, in such a case where a large amount of data which should be stored in the storage unit is generated in a continuous manner, data overflows may occur for any reasons other than the storage capacity of this storage unit. As a result, there are some possibilities that the data cannot be stored in the storage unit. For instance, if small-size data is continuously generated from a data source, or small-size data is generated from a large amount of data source, then a center side which processes those generated data is required to process the small-size data in the continuous manner. As a consequence, if an amount of data (hereinafter, referred to as “data flow rate”) which should be processed per unit time exceeds another amount of data (hereinafter, referred to as “processable flow rate”) per unit time, which can be actually processed according to storable or transferable speed of the data, then this storage unit must discard such data whose amount exceeds the processable flow rate. It should be noted that under the normal condition, a buffer is provided to the storage unit so as to absorb a difference between the data flow rate and the processable flow rate. However, when a data flow rate largely exceeds the processable flow rate, this storage unit cannot store the overflown data even in this buffer, and hence, the storage unit must discard such data which cannot be stored in the buffer, or other data which has already been stored in this buffer.
It should also be noted that in the data storage unit described in JP 2003-006005 A, the storage medium to which the data should be stored is selected by considering the capacity of the hard disk drive employed in this data storage unit, or considering the security thereof. However, the data storage unit does not consider the process performance of the selected storage medium. As a consequence, in such a case where this data storage unit is required to process the small-size and high priority data in the continuous manner, there are some possibilities that the data storage unit cannot store the data, because the process performance of the storage medium cannot accept this data storing operation.
Further, JP 2006-006005 A has described the method of controlling storing destinations of the data in view of the data priorities thereof. However, no concrete description has been made in that data having which data priority is stored in which storage medium. Also, since the above-mentioned operation for storing the data is not formularized, it is practically difficult to describe such a rule that the data priority of the data are defined with respect to the storage media to which the data should be stored in correspondence with each other.
On the other hand, the method described in “An optimization method for multiple persistency requirements on stream management system” by Yamada et al. 2007, Japanese Electronic Information Communication Institute Data Engineering Workshop (DEWS) has such a purpose that the processing itself is optimized, but never describes such a system for controlling a storing destination according to an amount of data. In other words, according to the method described in “An optimization method for multiple persistency requirements on stream management system” by Yamada et al. 2007, Japanese Electronic Information Communication Institute Data Engineering Workshop (DEWS), the algorithm is optimized by considering such an aspect of whether the contents of the processings can be commonly treated in the process flow. However, this method never considers how to control the storing destinations of the data when there are some possibilities that the data flow rate largely exceeds the processable flow rate.